Biostat 203B Homework 3

Due Feb 23 @ 11:59PM

Display machine information for reproducibility:

sessionInfo()

R version 4.1.2 (2021-11-01)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8       
 [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8   
 [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C          
[10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] htmlwidgets_1.6.4 compiler_4.1.2    fastmap_1.1.1     cli_3.6.2        
 [5] tools_4.1.2       htmltools_0.5.7   rstudioapi_0.15.0 yaml_2.3.8       
 [9] rmarkdown_2.25    knitr_1.45        jsonlite_1.8.8    xfun_0.41        
[13] digest_0.6.33     rlang_1.1.2       evaluate_0.23    

Load necessary libraries (you can add more as needed).

library(arrow)

Attaching package: 'arrow'

The following object is masked from 'package:utils':

    timestamp

library(memuse)
library(pryr)
library(R.utils)

Loading required package: R.oo

Loading required package: R.methodsS3

R.methodsS3 v1.8.2 (2022-06-13 22:00:14 UTC) successfully loaded. See ?R.methodsS3 for help.

R.oo v1.25.0 (2022-06-12 02:20:02 UTC) successfully loaded. See ?R.oo for help.

Attaching package: 'R.oo'

The following object is masked from 'package:R.methodsS3':

    throw

The following objects are masked from 'package:methods':

    getClasses, getMethods

The following objects are masked from 'package:base':

    attach, detach, load, save

R.utils v2.12.3 (2023-11-18 01:00:02 UTC) successfully loaded. See ?R.utils for help.

Attaching package: 'R.utils'

The following object is masked from 'package:arrow':

    timestamp

The following object is masked from 'package:utils':

    timestamp

The following objects are masked from 'package:base':

    cat, commandArgs, getOption, isOpen, nullfile, parse, warnings

library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.5
✔ forcats   1.0.0     ✔ stringr   1.5.1
✔ ggplot2   3.4.4     ✔ tibble    3.2.1
✔ lubridate 1.9.3     ✔ tidyr     1.3.0
✔ purrr     1.0.2     

── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ purrr::compose()      masks pryr::compose()
✖ lubridate::duration() masks arrow::duration()
✖ tidyr::extract()      masks R.utils::extract()
✖ dplyr::filter()       masks stats::filter()
✖ dplyr::lag()          masks stats::lag()
✖ purrr::partial()      masks pryr::partial()
✖ dplyr::where()        masks pryr::where()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

Display your machine memory.

memuse::Sys.meminfo()

Totalram:  7.657 GiB 
Freeram:   6.825 GiB 

In this exercise, we use tidyverse (ggplot2, dplyr, etc) to explore the MIMIC-IV data introduced in homework 1 and to build a cohort of ICU stays.

Q1. Visualizing patient trajectory

Visualizing a patient’s encounters in a health care system is a common task in clinical data analysis. In this question, we will visualize a patient’s ADT (admission-discharge-transfer) history and ICU vitals in the MIMIC-IV data.

Q1.1 ADT history

A patient’s ADT history records the time of admission, discharge, and transfer in the hospital. This figure shows the ADT history of the patient with subject_id 10001217 in the MIMIC-IV data. The x-axis is the calendar time, and the y-axis is the type of event (ADT, lab, procedure). The color of the line segment represents the care unit. The size of the line segment represents whether the care unit is an ICU/CCU. The crosses represent lab events, and the shape of the dots represents the type of procedure. The title of the figure shows the patient’s demographic information and the subtitle shows top 3 diagnoses.

Do a similar visualization for the patient with subject_id 10013310 using ggplot.

Hint: We need to pull information from data files patients.csv.gz, admissions.csv.gz, transfers.csv.gz, labevents.csv.gz, procedures_icd.csv.gz, diagnoses_icd.csv.gz, d_icd_procedures.csv.gz, and d_icd_diagnoses.csv.gz. For the big file labevents.csv.gz, use the Parquet format you generated in Homework 2. For reproducibility, make the Parquet folder labevents_pq available at the current working directory hw3, for example, by a symbolic link. Make your code reproducible.

ADT, admission, discharge, transfer lab, labevents (charttime) lab measurements procedures, procedures_icd.csv.gz

just add layers with + operator, just specify different data source

Answer:

ln -s ../hw2/labevents_parquet ./labevents_pq

ls -l ./labevents_pq/

total 2021528
lrwxrwxrwx 1 jasonduanchengwu jasonduanchengwu         24 Feb 11 11:25 labevents_parquet -> ../hw2/labevents_parquet
-rw-r--r-- 1 jasonduanchengwu jasonduanchengwu 2070040265 Feb  9 13:14 part-0.parquet

rm(list = ls())

# sid = 10001217
sid = 10013310
patient = read_csv("~/mimic/hosp/patients.csv.gz") %>%
  filter(subject_id == sid)

adt = read_csv("~/mimic/hosp/transfers.csv.gz") %>%
  filter(subject_id == sid & eventtype != "discharge")

admission = read_csv("~/mimic/hosp/admissions.csv.gz") %>%
  filter(subject_id == sid)

diagnoses = read_csv("~/mimic/hosp/diagnoses_icd.csv.gz") %>%
  filter(subject_id == sid) 

lab_diagnoses = diagnoses %>%
  head(3)

d_diagnoses = read_csv("~/mimic/hosp/d_icd_diagnoses.csv.gz")
lab_diagnoses = left_join(lab_diagnoses, d_diagnoses)

procedures = read_csv("~/mimic/hosp/procedures_icd.csv.gz") %>%
  filter(subject_id == sid)

labevents = open_dataset("labevents_pq", format = "parquet") %>%
  filter(subject_id %in% c(sid))

labevents = as_tibble(labevents)
  
d_procedures = read_csv("~/mimic/hosp/d_icd_procedures.csv.gz")
procedures = left_join(procedures, d_procedures)

procedures = procedures %>%
  mutate(chartdate = as.POSIXct(chartdate))

pid = str_c("Patient ", sid)
page = str_c(patient$anchor_age, " years old")
prace = tolower(admission$race)

# setting seed for reproducibility
set.seed(2)

Q1.1 = ggplot() +
  geom_point(data = procedures,
             aes(x = chartdate,
                 y = "Procedure",
                 shape = long_title),
             size = 3,
             position = position_jitter(width = 0, height = 0.3)) +
  scale_shape_manual(values = c(1:20))+
  geom_point(data = labevents,
           aes(x = charttime,
           y = "Lab"), 
           shape = 3,
           show.legend = F,
           position = position_jitter(width = 0,height = 0.2),
           alpha = 0.5) +
  geom_segment(data = adt,
             aes(x = intime,
                 xend = outtime,
                 y = "ADT",
                 yend = "ADT",
                 color = careunit,
                 linewidth = str_detect(careunit, "(ICU|CCU)"))) +
  labs(y = "", x = "Calendar Time",
     title = str_c(pid, patient$gender, page, prace, sep = ", "),
     subtitle = str_c(lab_diagnoses$long_title, collapse = "\n")) +
  theme_bw() + 
  scale_y_discrete(limits = c("Procedure", "Lab","ADT"))+
  theme(legend.position = "bottom", legend.box = "vertical", 
        legend.text = element_text(size = 6),
        legend.key.size = unit(0.2, "cm"),
        legend.title = element_text(size=6)) +
  guides(linewidth = "none",
         shape = guide_legend(order = 1, title = "Procedure", nrow = 5),
         color = guide_legend(order = 2, title = "Care Unit"))

ggsave(filename = "Q1.1.png", plot = Q1.1, path = "./", 
       width = 10, height = 5, units = c("in"))

Q1.2 ICU stays

ICU stays are a subset of ADT history. This figure shows the vitals of the patient 10001217 during ICU stays. The x-axis is the calendar time, and the y-axis is the value of the vital. The color of the line represents the type of vital. The facet grid shows the abbreviation of the vital and the stay ID.

Do a similar visualization for the patient 10013310.

Answer:

gunzip -c ~/mimic/icu/chartevents.csv.gz > chartevents.csv

library(arrow)
library(dplyr)

# write data as parquet format
chartevents_p = open_dataset("chartevents.csv", format = "csv")
write_dataset(chartevents_p, "chartevents_pq")

rm(list = ls())

library(arrow)
library(tidyverse)
library(lubridate)
library(ggplot2)
library(stringr)

# sid = 10001217
sid = 10013310

d_items = read_csv("~/mimic/icu/d_items.csv.gz")

Rows: 4014 Columns: 9
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (6): label, abbreviation, linksto, category, unitname, param_type
dbl (3): itemid, lownormalvalue, highnormalvalue

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

items = d_items %>%
  filter(itemid %in% c(220045, 220179, 220180, 220210, 223761)) %>%
  select(itemid, abbreviation)

chart_temp = open_dataset("chartevents_pq", format = "parquet")

# select columns and filter using dplyer
chartevents_pq = chart_temp %>%
  filter(subject_id %in% sid, 
         itemid %in% c(220045, 220179, 220180, 220210, 223761))

chartevents = as_tibble(chartevents_pq)

icu_data = left_join(chartevents, items)

Joining with `by = join_by(itemid)`

# Plot
Q1.2 = ggplot(icu_data, aes(x = charttime, y = valuenum, color = abbreviation, group = abbreviation)) +
  geom_point() +
  geom_line() +
  facet_grid(abbreviation ~ stay_id, scales = "free") +
  scale_x_datetime(guide = guide_axis(n.dodge = 2)) +
  labs(title = str_c("Patient ", sid, " ICU stays - Vitals"),
       x = "",
       y = "") +
  theme_light() +
  theme(legend.position = "none")

# Save plot so that we can control Dim
ggsave(filename = "Q1.2.png", plot = Q1.2, path = "./", 
       width = 12, height = 6, units = c("in"))

rm(list = ls())

Q2. ICU stays

icustays.csv.gz (https://mimic.mit.edu/docs/iv/modules/icu/icustays/) contains data about Intensive Care Units (ICU) stays. The first 10 lines are

zcat < ~/mimic/icu/icustays.csv.gz | head

subject_id,hadm_id,stay_id,first_careunit,last_careunit,intime,outtime,los
10000032,29079034,39553978,Medical Intensive Care Unit (MICU),Medical Intensive Care Unit (MICU),2180-07-23 14:00:00,2180-07-23 23:50:47,0.4102662037037037
10000980,26913865,39765666,Medical Intensive Care Unit (MICU),Medical Intensive Care Unit (MICU),2189-06-27 08:42:00,2189-06-27 20:38:27,0.4975347222222222
10001217,24597018,37067082,Surgical Intensive Care Unit (SICU),Surgical Intensive Care Unit (SICU),2157-11-20 19:18:02,2157-11-21 22:08:00,1.1180324074074075
10001217,27703517,34592300,Surgical Intensive Care Unit (SICU),Surgical Intensive Care Unit (SICU),2157-12-19 15:42:24,2157-12-20 14:27:41,0.9481134259259258
10001725,25563031,31205490,Medical/Surgical Intensive Care Unit (MICU/SICU),Medical/Surgical Intensive Care Unit (MICU/SICU),2110-04-11 15:52:22,2110-04-12 23:59:56,1.338587962962963
10001884,26184834,37510196,Medical Intensive Care Unit (MICU),Medical Intensive Care Unit (MICU),2131-01-11 04:20:05,2131-01-20 08:27:30,9.171817129629629
10002013,23581541,39060235,Cardiac Vascular Intensive Care Unit (CVICU),Cardiac Vascular Intensive Care Unit (CVICU),2160-05-18 10:00:53,2160-05-19 17:33:33,1.3143518518518518
10002155,20345487,32358465,Medical Intensive Care Unit (MICU),Medical Intensive Care Unit (MICU),2131-03-09 21:33:00,2131-03-10 18:09:21,0.8585763888888889
10002155,23822395,33685454,Coronary Care Unit (CCU),Coronary Care Unit (CCU),2129-08-04 12:45:00,2129-08-10 17:02:38,6.178912037037037

Q2.1 Ingestion

Import icustays.csv.gz as a tibble icustays_tble.

Answer:

library(tidyverse)
icustay_tble = read_csv("~/mimic/icu/icustays.csv.gz")

Rows: 73181 Columns: 8
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (2): first_careunit, last_careunit
dbl  (4): subject_id, hadm_id, stay_id, los
dttm (2): intime, outtime

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

icustay_tble = as_tibble(icustay_tble)

Q2.2 Summary and visualization

How many unique subject_id? Can a subject_id have multiple ICU stays? Summarize the number of ICU stays per subject_id by graphs.

Answer: The number of unique subject_id is shown in the code below and you can have multiple stays per id.

(unique_subject_id_count = icustay_tble %>%
  summarize(unique_subject_id_count = n_distinct(subject_id)) %>%
  pull(unique_subject_id_count))

[1] 50920

(icutble = icustay_tble %>%
  count(subject_id))

# A tibble: 50,920 × 2
   subject_id     n
        <dbl> <int>
 1   10000032     1
 2   10000980     1
 3   10001217     2
 4   10001725     1
 5   10001884     1
 6   10002013     1
 7   10002155     3
 8   10002348     1
 9   10002428     4
10   10002430     1
# ℹ 50,910 more rows

ggplot(data = icutble, aes(x = n)) +
  geom_bar(color = "black") +
  labs(title = "number of ICU stays per subject_id",
       x = "Number of ICU stays",
       y = "Number of corresponding patients") + 
  theme_bw()

rm(list = ls())

Q3. admissions data

Information of the patients admitted into hospital is available in admissions.csv.gz. See https://mimic.mit.edu/docs/iv/modules/hosp/admissions/ for details of each field in this file. The first 10 lines are

zcat < ~/mimic/hosp/admissions.csv.gz | head

subject_id,hadm_id,admittime,dischtime,deathtime,admission_type,admit_provider_id,admission_location,discharge_location,insurance,language,marital_status,race,edregtime,edouttime,hospital_expire_flag
10000032,22595853,2180-05-06 22:23:00,2180-05-07 17:15:00,,URGENT,P874LG,TRANSFER FROM HOSPITAL,HOME,Other,ENGLISH,WIDOWED,WHITE,2180-05-06 19:17:00,2180-05-06 23:30:00,0
10000032,22841357,2180-06-26 18:27:00,2180-06-27 18:49:00,,EW EMER.,P09Q6Y,EMERGENCY ROOM,HOME,Medicaid,ENGLISH,WIDOWED,WHITE,2180-06-26 15:54:00,2180-06-26 21:31:00,0
10000032,25742920,2180-08-05 23:44:00,2180-08-07 17:50:00,,EW EMER.,P60CC5,EMERGENCY ROOM,HOSPICE,Medicaid,ENGLISH,WIDOWED,WHITE,2180-08-05 20:58:00,2180-08-06 01:44:00,0
10000032,29079034,2180-07-23 12:35:00,2180-07-25 17:55:00,,EW EMER.,P30KEH,EMERGENCY ROOM,HOME,Medicaid,ENGLISH,WIDOWED,WHITE,2180-07-23 05:54:00,2180-07-23 14:00:00,0
10000068,25022803,2160-03-03 23:16:00,2160-03-04 06:26:00,,EU OBSERVATION,P51VDL,EMERGENCY ROOM,,Other,ENGLISH,SINGLE,WHITE,2160-03-03 21:55:00,2160-03-04 06:26:00,0
10000084,23052089,2160-11-21 01:56:00,2160-11-25 14:52:00,,EW EMER.,P6957U,WALK-IN/SELF REFERRAL,HOME HEALTH CARE,Medicare,ENGLISH,MARRIED,WHITE,2160-11-20 20:36:00,2160-11-21 03:20:00,0
10000084,29888819,2160-12-28 05:11:00,2160-12-28 16:07:00,,EU OBSERVATION,P63AD6,PHYSICIAN REFERRAL,,Medicare,ENGLISH,MARRIED,WHITE,2160-12-27 18:32:00,2160-12-28 16:07:00,0
10000108,27250926,2163-09-27 23:17:00,2163-09-28 09:04:00,,EU OBSERVATION,P38XXV,EMERGENCY ROOM,,Other,ENGLISH,SINGLE,WHITE,2163-09-27 16:18:00,2163-09-28 09:04:00,0
10000117,22927623,2181-11-15 02:05:00,2181-11-15 14:52:00,,EU OBSERVATION,P2358X,EMERGENCY ROOM,,Other,ENGLISH,DIVORCED,WHITE,2181-11-14 21:51:00,2181-11-15 09:57:00,0

Q3.1 Ingestion

Import admissions.csv.gz as a tibble admissions_tble.

Answer:

library(tidyverse)
admissions_tble = read_csv("~/mimic/hosp/admissions.csv.gz")

admissions_tble = as_tibble(admissions_tble)

Q3.2 Summary and visualization

Summarize the following information by graphics and explain any patterns you see.

• number of admissions per patient
  
• admission hour (anything unusual?)
  
• admission minute (anything unusual?)
  
• length of hospital stay (from admission to discharge) (anything unusual?)

According to the MIMIC-IV documentation,

All dates in the database have been shifted to protect patient confidentiality. Dates will be internally consistent for the same patient, but randomly distributed in the future. Dates of birth which occur in the present time are not true dates of birth. Furthermore, dates of birth which occur before the year 1900 occur if the patient is older than 89. In these cases, the patient’s age at their first admission has been fixed to 300.

(admissions_num = admissions_tble %>%
  count(subject_id))

# A tibble: 180,733 × 2
   subject_id     n
        <dbl> <int>
 1   10000032     4
 2   10000068     1
 3   10000084     2
 4   10000108     1
 5   10000117     2
 6   10000248     1
 7   10000280     1
 8   10000560     1
 9   10000635     1
10   10000719     1
# ℹ 180,723 more rows

ggplot(data = admissions_num, aes(x = n)) +
  geom_bar(color = "black") +
  labs(title = "number of admissions per patient",
       x = "Number of admissions",
       y = "Number of corresponding patients") + 
  theme_bw()

ggplot(data = admissions_tble, aes(x = hour(admittime))) +
  geom_bar() + 
  labs(title = "Admission count based on admission hour",
       x = "Admission Hour",
       y = "Number of Patients") +
  theme_bw()

Answer: It seems that there is fewer people admitted from 1am to 1pm with the exception of 7am. Other whole hour periods seems to have around the same admission numbers.

ggplot(data = admissions_tble, aes(x = minute(admittime))) +
  geom_bar() + 
  labs(title = "Admission count based on admission minute",
       x = "Admission Minute",
       y = "Number of Patients") +
  theme_bw()

Answer: The amount of people based on admission minute seems to be quite the same besides for every 15 mins at 0,15,30 and 45. It is interesting to think about, I do not know what exactly caused this but could be that the ambulance or ICU unit becomes more available at these timings.

admission = admissions_tble %>%
  mutate(len_of_stay = 
           as.integer(difftime(dischtime, admittime, units = "hours")))

class(admission$len_of_stay)
class(admission$admittime)
class(hour(admission$admittime))

ggplot(data = admission, aes(x = len_of_stay)) +
  geom_bar(color = "black") + 
  labs(title = "Admission count based on length of stay (Hours)",
       x = "Length of stay (Hours)",
       y = "Number of Patients") +
  theme_bw()

Answer: Since that most people stays in the ICU for a shorter amount of time however there are people that stay for really long time, they could be patients with severe conditions that requires constant attention at all times causing them to stay in the ICU for a long time.

Q4. patients data

Patient information is available in patients.csv.gz. See https://mimic.mit.edu/docs/iv/modules/hosp/patients/ for details of each field in this file. The first 10 lines are

zcat < ~/mimic/hosp/patients.csv.gz | head

subject_id,gender,anchor_age,anchor_year,anchor_year_group,dod
10000032,F,52,2180,2014 - 2016,2180-09-09
10000048,F,23,2126,2008 - 2010,
10000068,F,19,2160,2008 - 2010,
10000084,M,72,2160,2017 - 2019,2161-02-13
10000102,F,27,2136,2008 - 2010,
10000108,M,25,2163,2014 - 2016,
10000115,M,24,2154,2017 - 2019,
10000117,F,48,2174,2008 - 2010,
10000178,F,59,2157,2017 - 2019,

Q4.1 Ingestion

Import patients.csv.gz (https://mimic.mit.edu/docs/iv/modules/hosp/patients/) as a tibble patients_tble.

rm(list = ls())

Answer:

patients_tble = read_csv("~/mimic/hosp/patients.csv.gz")

Q4.2 Summary and visualization

Summarize variables gender and anchor_age by graphics, and explain any patterns you see.

Answer:

ggplot(data = patients_tble, aes(x = gender, y = anchor_age)) +
  geom_violin(trim = F) +
  geom_boxplot(width = 0.1, outlier.colour = "red") +
  scale_x_discrete(labels = c("Female", "Male")) +
  labs(title = "Violin and boxplot of Anchor age based on Gender",
       x = "Gender", y = "Anchor Age") +
  theme_bw()

Answer: This plot is a combination of boxplot which shows the interquartile range of the anchor age of the patients based on their gender and violin plot showing the density distribution of anchor age. Seems that male patient has a slightly higher median anchor age, and there is a decreasing density with increasing anchor age. The density ditribution of anchor age seems to be similar between female and male patients.

Q5. Lab results

labevents.csv.gz (https://mimic.mit.edu/docs/iv/modules/hosp/labevents/) contains all laboratory measurements for patients. The first 10 lines are

zcat < ~/mimic/hosp/labevents.csv.gz | head

labevent_id,subject_id,hadm_id,specimen_id,itemid,order_provider_id,charttime,storetime,value,valuenum,valueuom,ref_range_lower,ref_range_upper,flag,priority,comments
1,10000032,,45421181,51237,P28Z0X,2180-03-23 11:51:00,2180-03-23 15:15:00,1.4,1.4,,0.9,1.1,abnormal,ROUTINE,
2,10000032,,45421181,51274,P28Z0X,2180-03-23 11:51:00,2180-03-23 15:15:00,___,15.1,sec,9.4,12.5,abnormal,ROUTINE,VERIFIED.
3,10000032,,52958335,50853,P28Z0X,2180-03-23 11:51:00,2180-03-25 11:06:00,___,15,ng/mL,30,60,abnormal,ROUTINE,NEW ASSAY IN USE ___: DETECTS D2 AND D3 25-OH ACCURATELY.
4,10000032,,52958335,50861,P28Z0X,2180-03-23 11:51:00,2180-03-23 16:40:00,102,102,IU/L,0,40,abnormal,ROUTINE,
5,10000032,,52958335,50862,P28Z0X,2180-03-23 11:51:00,2180-03-23 16:40:00,3.3,3.3,g/dL,3.5,5.2,abnormal,ROUTINE,
6,10000032,,52958335,50863,P28Z0X,2180-03-23 11:51:00,2180-03-23 16:40:00,109,109,IU/L,35,105,abnormal,ROUTINE,
7,10000032,,52958335,50864,P28Z0X,2180-03-23 11:51:00,2180-03-23 16:40:00,___,8,ng/mL,0,8.7,,ROUTINE,MEASURED BY ___.
8,10000032,,52958335,50868,P28Z0X,2180-03-23 11:51:00,2180-03-23 16:40:00,12,12,mEq/L,8,20,,ROUTINE,
9,10000032,,52958335,50878,P28Z0X,2180-03-23 11:51:00,2180-03-23 16:40:00,143,143,IU/L,0,40,abnormal,ROUTINE,

d_labitems.csv.gz (https://mimic.mit.edu/docs/iv/modules/hosp/d_labitems/) is the dictionary of lab measurements.

zcat < ~/mimic/hosp/d_labitems.csv.gz | head

itemid,label,fluid,category
50801,Alveolar-arterial Gradient,Blood,Blood Gas
50802,Base Excess,Blood,Blood Gas
50803,"Calculated Bicarbonate, Whole Blood",Blood,Blood Gas
50804,Calculated Total CO2,Blood,Blood Gas
50805,Carboxyhemoglobin,Blood,Blood Gas
50806,"Chloride, Whole Blood",Blood,Blood Gas
50808,Free Calcium,Blood,Blood Gas
50809,Glucose,Blood,Blood Gas
50810,"Hematocrit, Calculated",Blood,Blood Gas

We are interested in the lab measurements of creatinine (50912), potassium (50971), sodium (50983), chloride (50902), bicarbonate (50882), hematocrit (51221), white blood cell count (51301), and glucose (50931). Retrieve a subset of labevents.csv.gz that only containing these items for the patients in icustays_tble. Further restrict to the last available measurement (by storetime) before the ICU stay. The final labevents_tble should have one row per ICU stay and columns for each lab measurement.

icustay_tble = read_csv("~/mimic/icu/icustays.csv.gz") %>%
  select(subject_id, stay_id, intime)

labevents_tble = open_dataset("labevents_pq", format = "parquet") %>%
  select(subject_id, itemid, valuenum, storetime) %>%
  filter(itemid %in% c(50912, 50971, 50983, 50902, 
                     50882, 51221, 51301, 50931))

labitems = read_csv("~/mimic/hosp/d_labitems.csv.gz") %>%
  select(itemid, label)

labevents_tble = as_tibble(labevents_tble) %>%
  left_join(icustay_tble, ., by = "subject_id") %>%
  filter(storetime < intime) %>%
  group_by(subject_id, stay_id, itemid) %>%
  arrange(desc(storetime), .by_group = T) %>%
  slice(1) %>%
  ungroup() %>%
  left_join(., labitems, by = "itemid") %>%
  select(subject_id, stay_id, valuenum, label) %>%
  pivot_wider(names_from = label, values_from = valuenum)

labevents_tble

# A tibble: 68,467 × 10
   subject_id  stay_id Bicarbonate Chloride Creatinine Glucose Potassium Sodium
        <dbl>    <dbl>       <dbl>    <dbl>      <dbl>   <dbl>     <dbl>  <dbl>
 1   10000032 39553978          25       95        0.7     102       6.7    126
 2   10000980 39765666          21      109        2.3      89       3.9    144
 3   10001217 34592300          30      104        0.5      87       4.1    142
 4   10001217 37067082          22      108        0.6     112       4.2    142
 5   10001725 31205490          NA       98       NA        NA       4.1    139
 6   10001884 37510196          30       88        1.1     141       4.5    130
 7   10002013 39060235          24      102        0.9     288       3.5    137
 8   10002155 31090461          23       98        2.8     117       4.9    135
 9   10002155 32358465          26       85        1.4     133       5.7    120
10   10002155 33685454          24      105        1.1     138       4.6    139
# ℹ 68,457 more rows
# ℹ 2 more variables: Hematocrit <dbl>, `White Blood Cells` <dbl>

Hint: Use the Parquet format you generated in Homework 2. For reproducibility, make labevents_pq folder available at the current working directory hw3, for example, by a symbolic link.

Q6. Vitals from charted events

chartevents.csv.gz (https://mimic.mit.edu/docs/iv/modules/icu/chartevents/) contains all the charted data available for a patient. During their ICU stay, the primary repository of a patient’s information is their electronic chart. The itemid variable indicates a single measurement type in the database. The value variable is the value measured for itemid. The first 10 lines of chartevents.csv.gz are

zcat < ~/mimic/icu/chartevents.csv.gz | head

subject_id,hadm_id,stay_id,caregiver_id,charttime,storetime,itemid,value,valuenum,valueuom,warning
10000032,29079034,39553978,47007,2180-07-23 21:01:00,2180-07-23 22:15:00,220179,82,82,mmHg,0
10000032,29079034,39553978,47007,2180-07-23 21:01:00,2180-07-23 22:15:00,220180,59,59,mmHg,0
10000032,29079034,39553978,47007,2180-07-23 21:01:00,2180-07-23 22:15:00,220181,63,63,mmHg,0
10000032,29079034,39553978,47007,2180-07-23 22:00:00,2180-07-23 22:15:00,220045,94,94,bpm,0
10000032,29079034,39553978,47007,2180-07-23 22:00:00,2180-07-23 22:15:00,220179,85,85,mmHg,0
10000032,29079034,39553978,47007,2180-07-23 22:00:00,2180-07-23 22:15:00,220180,55,55,mmHg,0
10000032,29079034,39553978,47007,2180-07-23 22:00:00,2180-07-23 22:15:00,220181,62,62,mmHg,0
10000032,29079034,39553978,47007,2180-07-23 22:00:00,2180-07-23 22:15:00,220210,20,20,insp/min,0
10000032,29079034,39553978,47007,2180-07-23 22:00:00,2180-07-23 22:15:00,220277,95,95,%,0

d_items.csv.gz (https://mimic.mit.edu/docs/iv/modules/icu/d_items/) is the dictionary for the itemid in chartevents.csv.gz.

zcat < ~/mimic/icu/d_items.csv.gz | head

itemid,label,abbreviation,linksto,category,unitname,param_type,lownormalvalue,highnormalvalue
220001,Problem List,Problem List,chartevents,General,,Text,,
220003,ICU Admission date,ICU Admission date,datetimeevents,ADT,,Date and time,,
220045,Heart Rate,HR,chartevents,Routine Vital Signs,bpm,Numeric,,
220046,Heart rate Alarm - High,HR Alarm - High,chartevents,Alarms,bpm,Numeric,,
220047,Heart Rate Alarm - Low,HR Alarm - Low,chartevents,Alarms,bpm,Numeric,,
220048,Heart Rhythm,Heart Rhythm,chartevents,Routine Vital Signs,,Text,,
220050,Arterial Blood Pressure systolic,ABPs,chartevents,Routine Vital Signs,mmHg,Numeric,90,140
220051,Arterial Blood Pressure diastolic,ABPd,chartevents,Routine Vital Signs,mmHg,Numeric,60,90
220052,Arterial Blood Pressure mean,ABPm,chartevents,Routine Vital Signs,mmHg,Numeric,,

We are interested in the vitals for ICU patients: heart rate (220045), systolic non-invasive blood pressure (220179), diastolic non-invasive blood pressure (220180), body temperature in Fahrenheit (223761), and respiratory rate (220210). Retrieve a subset of chartevents.csv.gz only containing these items for the patients in icustays_tble. Further restrict to the first vital measurement within the ICU stay. The final chartevents_tble should have one row per ICU stay and columns for each vital measurement.

Answer:

icustay_tble = read_csv("~/mimic/icu/icustays.csv.gz") %>%
  select(subject_id, stay_id)

chartevents_tble = open_dataset("chartevents_pq", format = "parquet") %>%
  select(subject_id, itemid, charttime, valuenum, stay_id) %>%
  filter(itemid %in% c(220045, 220179, 220180, 223761, 220210))

items = read_csv("~/mimic/icu/d_items.csv.gz") %>%
  select(itemid, label)

chartevents_tble = as_tibble(chartevents_tble) %>%
  left_join(., items, by = "itemid") %>%
  left_join(icustay_tble, ., 
            by = c("subject_id", "stay_id" = "stay_id")) %>%
  group_by(subject_id, stay_id, label) %>%
  filter(charttime == min(charttime)) %>%
  distinct() %>%
  ungroup() %>%
  select(subject_id, stay_id, label, valuenum) %>%
  pivot_wider(names_from = label, values_from = valuenum)

chartevents_tble

# A tibble: 73,164 × 7
   subject_id  stay_id `Temperature Fahrenheit` Non Invasive Blood Pressure sy…¹
        <dbl>    <dbl>                    <dbl>                            <dbl>
 1   10000032 39553978                     98.7                               84
 2   10000980 39765666                     98                                150
 3   10001217 37067082                     98.5                              151
 4   10001217 34592300                     97.6                              167
 5   10001725 31205490                     97.7                               73
 6   10001884 37510196                     98.1                              180
 7   10002013 39060235                     97.2                              104
 8   10002155 32358465                     97.7                              109
 9   10002155 33685454                     95.9                              126
10   10002155 31090461                     96.9                              118
# ℹ 73,154 more rows
# ℹ abbreviated name: ¹​`Non Invasive Blood Pressure systolic`
# ℹ 3 more variables: `Non Invasive Blood Pressure diastolic` <dbl>,
#   `Heart Rate` <dbl>, `Respiratory Rate` <dbl>

Hint: Use the Parquet format you generated in Homework 2. For reproducibility, make chartevents_pq folder available at the current working directory, for example, by a symbolic link.

Q7. Putting things together

Let us create a tibble mimic_icu_cohort for all ICU stays, where rows are all ICU stays of adults (age at intime >= 18) and columns contain at least following variables

• all variables in icustays_tble
  
• all variables in admissions_tble
  
• all variables in patients_tble
• the last lab measurements before the ICU stay in labevents_tble
• the first vital measurements during the ICU stay in chartevents_tble

The final mimic_icu_cohort should have one row per ICU stay and columns for each variable.

Answer:

icustays_tble = read_csv("~/mimic/icu/icustays.csv.gz")
admissions_tble = read_csv("~/mimic/hosp/admissions.csv.gz")
patients_tble = read_csv("~/mimic/hosp/patients.csv.gz") %>%
  mutate(anchor_age_recalc = 
           ymd(paste(anchor_year, "-01-01")) - years(anchor_age))

mimic_icu_cohort = icustays_tble %>%
  distinct(subject_id, 
           hadm_id, 
           stay_id, 
           .keep_all = TRUE) %>% 
  left_join(admissions_tble, 
            by = c("subject_id", "hadm_id")) %>%
  left_join(patients_tble,
            by = "subject_id") %>%
  group_by(subject_id, 
           stay_id) %>%
  slice(1) %>%
  ungroup() %>% 
  left_join(labevents_tble, by = c("subject_id", "stay_id")) %>%
  left_join(chartevents_tble, by = c("subject_id", "stay_id")) %>% 
  mutate(age_intime = 
           trunc(time_length(difftime(intime, anchor_age_recalc), "years"))) %>%
  filter(age_intime >= 18) %>%
  select(-anchor_age_recalc)

mimic_icu_cohort

# A tibble: 73,181 × 41
   subject_id  hadm_id  stay_id first_careunit last_careunit intime             
        <dbl>    <dbl>    <dbl> <chr>          <chr>         <dttm>             
 1   10000032 29079034 39553978 Medical Inten… Medical Inte… 2180-07-23 14:00:00
 2   10000980 26913865 39765666 Medical Inten… Medical Inte… 2189-06-27 08:42:00
 3   10001217 27703517 34592300 Surgical Inte… Surgical Int… 2157-12-19 15:42:24
 4   10001217 24597018 37067082 Surgical Inte… Surgical Int… 2157-11-20 19:18:02
 5   10001725 25563031 31205490 Medical/Surgi… Medical/Surg… 2110-04-11 15:52:22
 6   10001884 26184834 37510196 Medical Inten… Medical Inte… 2131-01-11 04:20:05
 7   10002013 23581541 39060235 Cardiac Vascu… Cardiac Vasc… 2160-05-18 10:00:53
 8   10002155 28994087 31090461 Medical/Surgi… Medical/Surg… 2130-09-24 00:50:00
 9   10002155 20345487 32358465 Medical Inten… Medical Inte… 2131-03-09 21:33:00
10   10002155 23822395 33685454 Coronary Care… Coronary Car… 2129-08-04 12:45:00
# ℹ 73,171 more rows
# ℹ 35 more variables: outtime <dttm>, los <dbl>, admittime <dttm>,
#   dischtime <dttm>, deathtime <dttm>, admission_type <chr>,
#   admit_provider_id <chr>, admission_location <chr>,
#   discharge_location <chr>, insurance <chr>, language <chr>,
#   marital_status <chr>, race <chr>, edregtime <dttm>, edouttime <dttm>,
#   hospital_expire_flag <dbl>, gender <chr>, anchor_age <dbl>, …

Q8. Exploratory data analysis (EDA)

Summarize the following information about the ICU stay cohort mimic_icu_cohort using appropriate numerics or graphs:

• Length of ICU stay los vs demographic variables (race, insurance, marital_status, gender, age at intime)

Answer:

ggplot(data = mimic_icu_cohort, aes(y = los, x = race)) +
  geom_violin() +
  labs(title = "Length of ICU stay vs Race",
       x = "Race",
       y = "Length of ICU stay") +
  theme(axis.text.x = element_text(angle = 45, hjust = 1))

ggplot(data = mimic_icu_cohort, aes(y = los, x = as_factor(age_intime))) +
  geom_boxplot() +
  labs(title = "Length of ICU Stay vs Age at Intime", 
       x ="Length of Stay (days)", 
       y = "Age at Intime")

ggplot(mimic_icu_cohort, aes(x = los, y = insurance)) +
  geom_violin() +
  labs(title = "Length of ICU Stay by Insurance", 
       x = "Length of Stay", 
       y = "Insurance")

#plot with marital status
ggplot(mimic_icu_cohort, aes(x = marital_status, y = los)) +
  geom_violin() +
  labs(title = "Length of ICU Stay vs Marital Status",
       x = "Marital Status",
       y = "Length of Stay")

#plot with gender
ggplot(mimic_icu_cohort, aes(x = gender, y = los)) +
  geom_violin() +
  labs(title = "Length of ICU Stay vs Gender", 
       x = "Gender", 
       y = "Length of Stay")

• Length of ICU stay los vs the last available lab measurements before ICU stay

Answer:

mimic_icu = mimic_icu_cohort %>%
  gather(28:35, key = "key", 
         value = "value" ) %>%
  group_by(key)
  
ggplot(data = mimic_icu, aes(x = value, y = los)) +
  geom_point() +
  facet_wrap(~key, scales = "free") + 
  labs(title = "Length of ICU Stay vs Last Lab Measurements", 
       x = "Value", 
       y = "Length of Stay")

Warning: Removed 60686 rows containing missing values (`geom_point()`).

• Length of ICU stay los vs the average vital measurements within the first hour of ICU stay

Answer:

mimic_icu = mimic_icu_cohort %>%
  gather(36:40, key = "key", value = "value") %>%
  group_by(key)

ggplot(data = mimic_icu, aes(x = value, y = los)) +
  geom_point() +
  facet_wrap(~key, scales = "free") +
  labs(title = "Length of ICU Stay vs First Vital Measurements", 
       x = "Value", 
       y = "Length of Stay")

Warning: Removed 3261 rows containing missing values (`geom_point()`).

• Length of ICU stay los vs first ICU unit

Answer:

ggplot(data = mimic_icu_cohort, aes(x = first_careunit, y = los)) +
  geom_violin() +
  labs(title = "Length of ICU Stay vs First ICU Unit", 
       x = "First ICU Unit", 
       y = "Length of Stay") +
  theme(axis.text.x = element_text(angle = 60, hjust = 1, size = 8))